There is a large interest in handling audio signals, and in particular multi-channel audio signals. Audio signals are achieved, either as direct recordings or generation of stereophonic sound or as retrieval of earlier stored representations of audio signals, and transmitted in some way to an end unit, such as a loudspeaker system or storage for audio signals. In digital signal systems, the audio signals are typically encoded before being transmitted, and decoded at the receiver side.
Parametric encoding is found to be very attractive, since the required bit rate for transmitting multi-channel data can be reduced significantly compared to mere waveform encodings. There are several examples of parametric encoding schemes in prior art.
Regardless of what transmission method is used, communications systems are typically associated with error-prone transmission channels, e.g. in wireless communication or through the Internet. There are several levels of combating erroneously received signals. Directly in the transmission layer, there are error handling routines, such as forward error correction (FEC) and retransmission schemes, which try to compensate for certain types of transmission-induced errors. However, some errors cannot fully be repaired by such transmission-error schemes and the decoders, in the case of digital signals, have to be configured to receive also corrupted signals or even cope with lost signal portions. Typically, the decoders will receive coded data corresponding to frames of the input signal and there is typically a flag indicating if the frame data is error-free or corrupted or lost, i.e. unusable. In case of unusable data, the decoder will not be able to decode and reconstruct the corresponding signal frame. Instead means for frame loss concealment will be deployed, rendering the loss as inaudible as possible.
In case of stereophonic or multi-channel audio signals the frame loss may specifically affect the stereo or multi-channel audio representation. E.g. if one of the transmitted channels is affected, the decoder may still be able to reconstruct the other channel, or, depending on the chosen equivalent representation, it may still be possible to reconstruct a monophonic signal. However, a sudden loss of one of the audio channels as well as e.g. the sudden change from a stereo to a mono signal will harm the perceived audio quality. An important part of audio codec error concealment is thus the mitigation of losses of stereophonic or multi-channel information.
Most signal loss concealment methods of prior art are directly connected to the type of encoding used during a transmission step. Depending on the type of audio codec, parametric or non-parametric, there are various ways to realize error concealment for audio codecs in general including those for stereophonic or multi-channel audio. Common for all of these are that they are performing concealment attempts during or in direct connection to the actual decoding process.
Non-parametric audio codecs will typically repeat or estimate e.g. by means of interpolation correctly received signal values in order to generate a substitution for the erroneous values. As an example, the U.S. Pat. No. 6,490,551 by Wiese et. al. teaches substituting lost spectral components by estimates (e.g. by interpolation) from corresponding components of the same or another (stereo) channel including time or frequency domain sampled values. The merit of this patent is that, as it claims, it maintains the stereophonic impression.
Another similar technique particularly for stereo signals is described in patent DE 3638922 according to which lost signal sections of one of the stereo channels are replaced by corresponding signal sections of the other channel.
Another similar technique particularly for stereo signals is described in the patent DE 3638922 according to which lost signal sections of one of the stereo channels are replaced by corresponding signal sections of the other channel.
Typical frame loss concealment for parametric audio codecs involves replacing an erroneous parameter by an earlier and correctly received corresponding parameter. This is a temporal technique widely used in speech codecs that is directly applicable for parametric audio codecs. It is described in detail e.g. in the 3GPP specification on error concealment of lost frames for the AMR speech codec, 3 GPP TS 26.091, clauses 6 and 7.
Patent EP 0 637 013 by Cluever describes a parametric frame loss concealment method for non-parametric monophonic speech codecs. The signal values from a correctly received speech frame are used to derive the parameters of a speech synthesis model. In case of a frame loss the missing speech frame is synthesized by applying that model using the parameters derived during the last valid speech frame. Such a technique could in principle be applied for error concealment in audio codecs, and in the multi-channel case channel by channel.
Among the loss concealment methods discussed above, most approaches restore the transmitted channels in multi-channel audio systems independently of each other, ignoring any statistical inter-channel dependencies. The methods of U.S. Pat. No. 6,490,551 and DE 3638922 do explicitly exploit inter-channel dependencies, but are limited as solutions of non-parametric codecs. Moreover, they employ a principle of repeating or interpolative estimation of correctly received signal values in order to generate a substitution for the erroneous values, which typically doesn't lead to the best perceptual quality.